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1. INTRODUCTION 


The space industry has developed many composite materials that have been designed to 
have high durability in proportion to their weights. Many of these materials have a likelihood for 
flaws that is higher than in traditional metals. There are also material coverings (such as paint) 
that develop flaws that may adversely affect the performance of the system in which they are 
used. Therefore, there is a need to monitor the soundness of composite structures. To meet this 
monitoring need, many nondestructive evaluation (NDE) systems have been developed. An NDE 
system is designed to detect material flaws and make flaw measurements without destroying the 
inspected item. Also, the detection operation is expected to be performed in a rapid manner in a 
field or production environment. 

Within the last few years, several video-based optical NDE methodologies have been intro- 
duced. Some of the most recent of these methodologies are shearography, holography, thermog- 
raphy, and video image correlation. A detailed description of these may be found in Chu et al. 
(1985), Hung (1982), and Russell and Sutton (1989). 

The NDE Branch of the Materials and Processes Laboratory at Marshall Space Flight 
Center has contracted to purchase video optical NDE equipment. Therefore, that Branch is now 
interested in qualifying the performance capability of this equipment. Qualification standards are 
found in MIL-STD (1989, draft No. 2) which gives the probability of detection (POD) curve as 
the primary qualification measurement. This curve summarizes the POD of a particular equip- 
ment type for a wide range of flaw sizes. 

Section 2 will discuss what techniques are now available to estimate the POD curve from 
sample data. Section 3 will develop needed extensions for shortcomings in POD analysis. Sec- 
tion 4 shows how these extensions may be used to solve problems in the NDE Branch. 


2. METHODS FOR COMPUTING POD 

Figure 1 shows an example of a POD curve along with a lower 95-percent confidence 
boundary. The horizontal axis represents flaw size a and the vertical axis expresses probability as 
a percentage. As expected, the POD increases with flaw size. The darker curve is labeled POD 
and is sometimes called the 50-percent confidence boundary or the estimated mean POD. The 
lower lighter curve is called (in this figure), the lower 95-percent confidence boundary. 



a axis 

Figure 1 
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When qualifying an NDE equipment system, two major tasks must be performed. They are: 
(a) use sample data (perhaps from a demonstration test) to estimate the POD curve, and (b) 
determine a confidence boundary on the estimated POD curve. MIL- STD specifies a 95-percent 
confidence boundary. 

Researchers have always classified POD(a) as a random variable that depends on a . This 
classification means that if the probability distribution of POD (a) s known, then mean POD and 
confidence boundaries are easily computed. Direct approaches used to identify the probability 
distribution of POD(a) have met with varying degrees of success. At least three different direct 
approaches are discussed by Berens and Hovey (1984). They also described alternative 
approaches that avoid the difficulties normally encountered with direct POD(a) distribution 
identification procedures. One is the pass/fail method and the other is labeled as the a-hat 
method. 

Pass/Fail Method 

The first method used to estimate the POD curve is described in Packman et al. (1976) and 
is based on the binomial distribution. To describe this method, let us assume that there are Ni 
identically fabricated flaws for each fixed flaw size a u An NDE system examines the Ni flaws 

and detects m of them. Binomial theory tells us that P, = ^ is an estimate of POD for flaw size 

a*-. Packman uses a slightly different definition for P, and uses a collection of P/’s, i = 1,2,.., k, to 
estimate a POD curve and its confidence boundary. A NASA supported study (Yee et al. (1976)) 
improves Packman’s method by using a smoothing procedure. The improved method appears in 
the USAF POD software system as subroutine FF. The NDE Branch at MSFC uses this software 
system (see Berens et al. (1988)). 

A-hat Method 

An alternative to the direct characterization of the distribution of POD(a) is to find a proba- 
bility distribution for the output signal of the NDE equipment. In fact, if we let a denote equip- 
ment response to the examination of a flaw of size a, experimentation by Berens et al. (1984) has 
shown that a and a are related by 


In a = (XQ+cXilna + e , (2.1) 

where e denotes a random error term. 

Berens also showed that In a has an approximate normal probability distribution. Hence, 
to summarize the probability behavior of equation (2.1), we say that e ~ N(0,cr 2 ), In a ~ 

N(fi(a),<J 2 ) where ji(a) = ccq+cci In a and NQ denotes the normal distribution. The value of 
POD(a) is computed by 

POD (a) = P(ln a> a th ) , (2.2) 

where a t h is called a threshold value and 
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(2.3) 


a th ~n(a)+Z p G . 

Here, Z p is the Pth percentile of the standard normal distribution. 

From sample to sample, sample estimates fl(a ) and a of parameters in equation (2.3) 
change in a random manner. The random behavior of /1(a) and a makes POD(a) a random vari- 
able. (It appears that a th changes from sample to sample. Actually, the probability distribution 
changes relative to a t h where a t h is fixed.) Hence, POD(a) as defined in equation (2.2) has a 
probability distribution which, if known, may be used to determine a confidence boundary for 
POD(a). However, Cheng and lies (1983) showed that if /i(a) and c are estimated and their 
confidence region determined, a corresponding confidence boundary for POD(a) may be 
determined without actually knowing the probability distribution of POD(a). Their approach 
used Lagrange multipliers to find maximum and minimum values of a t h where cq, and <7 are 
restricted to some confidence region. The maximum and minimum values of a t h are used to find 
corresponding confidence boundaries for POD(a). 


3. SHORTCOMING OF POD 

The probability of false calls (POF) is the probability that an NDE system will indicate the 
presence of a flaw when a flaw does not exist. The POD(a) curve does not assess this situation. 
In fact, MIL-STD (1989) left POF evaluation as an open question. The next section uses the rela- 
tionship in equation (2.1) to recommend a computation procedure for POD and POF. 


4. GENERAL SOLUTION FOR POD. POF 


Let y - In a where a is the output of NDE equipment when a flaw exists. Let X ^ - In a 
where a is actual flaw size. Assume 


y = f(X i, #i) + £i , (4.1) 

where y is normal, X \ is a vector that contains xi as a first component, and e x ~ N(0,6) . In a 
similar manner, let w denote the logarithm of the output of NDE equipment when flaw does not 
exist. Assume w = g(X 1 ,& 2 )+£ 2 where w has normal distribution and e 2 ~ N(0,cr 2 ) . In func- 
tions/ and g, parameters a\, 1 X 2 , and care to be estimated from sample data. POD and POF are 
based on equation (2.2). That is 


POD(a) = P(y > a t h) , (4.2) 

and 


POF(a) = P(w > a t h) . (4.3) 

After parameters are estimated, confidence bounds for POD(a) and POF(a) may be determined 
by procedures outlined in section 2. 
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Problem 


Qualification standards require that the camera used in video optical equipment be qualified 
at a fixed distance between the camera lens and the object being examined. However, the NDE 
Branch of the Materials and Processes Laboratory is interested in the POD performance of this 
equipment at the tested distance as well as at untested distances. The solution to this problem 
requires the use of equation (4.2) which has the ability to predict POD performance of NDE 
equipment. 


5. RECOMMENDATION 

A solution to the problem posed above may be obtained by using equations (4.1) and (4.2). 
In this discussion, original variables of equation (2.1) will be used instead of the general vector 
variables of equation (4.1). That is, using equation (2.1) along with d which represents the dis- 
tance of the camera lens from the examined object, a may be related to a and d by 

In a = a 0 +a l In a+a 2 d+e 1 = /j (a,d)+e l , (5.1) 


or 


In a = (Xq+Ci d In a+e 2 = / 2 (<z,d)+£ 2 * (5.2) 

Data inspection will provide a more realistic relationship. Furthermore, analysis of experimental 
data will determine if d in equations (5.1) or (5.2) is a significant predictor of In a . If d is judged 
to be a significant variable, then POD(a) and its confidence bounds may be determined for any 
fixed distance d. Confidence boundaries are determined as discussed in section 2. However, if 
the functions f\ or f 2 are nonlinear in their parameters, the confidence boundary computation 
procedure will depend on the nonlinear function. Actual data for this problem will be analyzed at 
a later date. 
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